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Abstract: This paper considers demand side management in smart power grid systems contain- 
ing significant numbers of tlrermostatically controlied loads suclr as air conditioning systems, 
heat pumps, etc. Recent studies have shown that the overall power consumption of such systems 
can be regulated up and down centrally by broadcasting small setpoint change commands 
without significantly impacting consumer comfort. However, sudden simultaneous setpoint 
changes induce undesirable power consumption oscillations due to sudden synchronization of 
the on/off cycles of the individual units. In this paper, we present a novel algorithm for 
counter-acting these unwanted oscillations, which requires neither central management of the 
individual units nor communication between units. We present a formal proof of convergence of 
homogeneous populations to desynchronized status, as well as simulations that indicate that the 
algorithm is able to effectively dampen power consumption oscillations for both homogeneous 
and heterogeneous populations of thermostatically controlled loads. 



1. INTRODUCTION 

With growing penetration of renewable energy sources in 
modern power grids, demand side management has been 
gaining attention as a means of achieving better balancing 
between supply and demand (Mohsenian-Rad et al. [2010], 
Strbac [2008], Short et al. [2007]). Indeed, it appears 
that the higher intermittency and lack of dispatchability 
associated with increased dependence on renewable energy 
sources can be taken care of more effectively by electrical 
loads than by conventional generators, which are typically 
not designed for fast up- and down regulation (Strbac 
[2008], Klobasa [2010]). 

Various technologies are currently being considered in the 
context of demand side management; coordinated charging 
of batteries, e.g., in electric vehicles (Mets et al. [2010]), 
deliberate scheduling of loads with flexible deadlines (Pe- 
tersen et al. [2012]) as well as allowing local consumers 
with slow dynamics (large time constants) to store more 
or less energy at convenient times and thereby adjusting 
the momentary consumption (e.g., Moslehi and Kumar 
[2010]), among others. In particular, so-called thermostati- 
cally controlled loads (TCLs), such as deep freezers, refrig- 
erators, local heat pumps etc., are showing great potential 
in this context, since they account for a large volume of 
consumption in most countries with significant renewable 
penetration — for instance, as of 2009, about 87 percent of 
all US homes were equipped with air conditioning. ^ 
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Thus, at least in theory, manipulating the operating condi- 
tions of large populations of units slightly while avoiding 
discomfort to end users appears to be attractive, given 
that it can be achieved simply by broadcasting setpoint 
changes. Control strategies based on this principle were 
considered in CaUaway [2009], Kundu et al. [2011], Per- 
fumo et al. [2012] and Bashash and Fathy [2012], among 
others. In these strategies, subtle changes in the thermo- 
stat setpoint temperature (less than a degree Celcius) are 
transmitted to all the participating consumers simultane- 
ously. Such small variations in the thermostat setpoints are 
expected to remain almost unnoticed by the customers, 
but if a sufficiently large number of units are shifted in 
the same direction, the overall power consumption can be 
shifted quickly by significant amounts, allowing to com- 
pensate for power production fluctuations in the grid. 

There is an intrinsic problem with this approach, how- 
ever. When the setpoint is changed concurrently in a 
large number of TCLs, their on/off cycles tend to become 
synchronized, which leads to large unwanted fluctuations 
(damped oscillations) in the overall power consumption 
Callaway [2009], Kundu et al. [2011]. If the entire popu- 
lation of TCLs consists of identical units (homogeneous) , 
it is possible to counteract the fluctuations by means of 
a centralized control law. In the much more realistic case 
of non-identical (heterogeneous) units, on the other hand, 
the fluctuations are far harder to remove using central- 
ized control strategies. To the authors' knowledge, the 
only solution strategy presented in the literature so far is 
Kundu and Sinitsyn [2012], where a strategy in which the 
individual TCLs deliberately slow down their transition in 
order to avoid synchronization was proposed. 

In this paper, we propose a novel, decentralized algo- 
rithm for avoiding synchronization without having to ac- 



tively communicate between units. The algorithm pro- 
posed here is fundamentally different from the one pre- 
sented in Kundu and Sinitsyn [20f 2]) in that it first adjusts 
the temperature bands in which the TCLs operate to 
achieve early (randomized) desynchronization, and then 
makes use of a technique inspired by contention-based 
media access protocols to adaptively adjust the on/off 
cycles to achieve as little synchronization among units 
as possible. We present a formal proof of convergence of 
homogeneous populations to this desynchronized status, 
as well as simulations that indicate that the algorithm is 
able to effectively dampen power consumption oscillations 
for both homogeneous and heterogeneous populations of 
thermostatically controlled loads. 

The outline of the rest of the paper is as follows. Section 
2 first presents the simplified TCL model we shall em- 
ploy, along with the synchronization issue. Section 3 then 
presents the proposed algorithm along with the aforemen- 
tioned convergence result. Section 4 illustrates the feasi- 
bility of the approach through two simulation examples. 
Finally, Section 5 offers some concluding remarks. 

2. PROBLEM SETUP 

We consider a population of A'^ individual temperature 
controlled loads. The TCLs are modeled in a generic 
manner, which is deliberately kept as simple as possible. 
Nonetheless, as will be illustrated with a few simulation 
examples, the behavior of a large population of simple 
units can be fairly complex. 

2.1 Model 

The individual TCLs are modeled as follows. Let the 
internal and ambient temperatures of the volumes (living 
spaces, refrigerators, cold storages etc.) affected by the 
action of the heating/cooling hardware of the i'th con- 
sumer be denoted 6i and 6*00,4, respectively, and assume 
that the hardware is purely on/off- regulated. A simple 
model for the internal temperature can then be formulated 
as Malhame and Chong [1985], Callaway [2009], Bashash 
and Fathy [2012]: 

^,-e,{t)-s,{t)R,P{), (1) 

if s,(t-) = lAT,(i)<T,„i„,, 

if s,(t-) -OAT,(t) >T,„ax,. (2) 
otherwise 

for i — 1,2, . . . , N , where Ci is the thermal capacitance of 
the consumer, Ri is the corresponding thermal resistance 
and Pi is the (constant) heating/cooling power supplied 
by the hardware when switched on. Si £ {0, 1} is a 
binary switching variable that determines whether or not 
the hardware is turned on; basically, it switches status 
whenever the internal temperature encounters the limits of 
a pre-set temperature span [0min,i, ^max,i] CM. {t~ = t — e, 
where e is an infinitesimal positive value). Note that both 
heating and cooling systems can be modeled by (4)-(2), 
by choosing the sign of the term Si(t)RiPi appropriately. 

The temperature limits 9 
j'th consumer's setpoint O^p^i 





through the relations 



Fig. 1. Temperature and power consumption of a single 
TCL 



where A is the width of the temperature interval. Further- 
more, the cumulative power consumption of the population 
of TCLs at any given time t can be computed as 

where r]i is the coefficient of performance for the z'th 
heating/cooling unit. 

Figure 1 illustrates the characteristic behavior of a single 
TCL. The parameters are chosen as i? = 2°C/kW, C = 5 
kWh/°C, P = 14 kW, r? = 2.5, 0sp = 20°C, A = 1°C 
and 9ao = 28°C, respectively. The temperature setpoint is 
changed to 9sp — 20.5°C at time i = 10 h, resulting in a 
slightly longer period in which the cooling power is turned 
off, until the new operating interval is reached. 

2.2 Population behavior 

As illustrated in Callaway [2009], Perfumo et al. [2012], 
Bashash and Fathy [2012] and others, the power consump- 
tion of populations of units with the individual dynamics 
given above can be manipulated via broadcasts of small 
setpoint changes. Figure 2 shows the behavior of a pop- 
ulation of 10,000 homogeneous (identical) TCLs with the 
same set of parameters as in figure 1 (only the temperature 
profiles of 25 TCLs are shown in the figure). The devices 
are started at random temperatures unifomly distributed 
within [^min, 6'max], and their on/off cycles are thus desyn- 
chronized, yielding a fairly smooth power consumption 
trajectory. At time t = 10 h, all the TCLs are subjected to 
a common step in the setpoint of 0.5°C. As can be seen, 
the overall power consumption drops by an amount pro- 
portional to the size of the step, but significant oscillations 
appear in the power consumption due to synchronization 
of the individual units' on/off cycles. 

Figure 3 illustrates the behavior of a population of TCLs 
under the same simulation circumstances, but with Ci cho- 
sen from a normal distribution with mean 5 and spread 0.5 
kWh/°C (again, only the temperature profiles of 25 TCLs 
are shown in the figure). A step of 0.5°C in the setpoint 
is broadcast to all the TCLs at time t = 10 h. Notice how 
the power consumption is quite smooth before the step 
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Fig. 2. Temperature and power consumption of a popula- 
tion of 10,000 homogeneous TCLs 



Thus, we look for a low-complexity, decentralized algo- 
rithm that can be implemented locally in each TCL, and 
which satisfies the following requirements: 

Rl The algorithm may only use local information; this 
means that the TCLs may not communicate with 
each other, and labels identifying each individual 
TCL may not be pre-assigned 

R2 Communication with the power supply utility must 
be limited to broadcast from the utility; no commu- 
nication originating from the TCLs is allowed 

R3 The general operation of each individual TCL may 
not be altered in a manner that is detrimental to 
the user comfort; for example, the unit may not be 
deliberately kept turned off or on long enough for the 
temperature to leave the interval [^min, ^max] 

3. DESYNCHRONIZATION ALGORITHM 
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Fig. 3. Temperature and power consumption of a popula- 
tion of 10,000 heterogeneous TCLs 

because the on/off cycles of the TCLs are desynchronized, 
whereas immediately after the step the TCLs become 
synchronized and the power consumption again exhibits 
large fluctuations, which in this case die out slowly. The 
oscillations die out faster if the parameters vary more (e.g., 
if Ci is chosen from a distribution with larger spread); 
furthermore, the amplitude is relatively larger for larger 
populations of units. However, qualitatively the behavior 
remains the same. The oscillations are known as parasitic 
oscillations and are hard to remove via centralized control 
signals (Kundu and Sinitsyn [2012]). 

2.3 Problem formulation 

In order to reduce the aforementioned oscillations in power 
consumption after the setpoint change broadcasts, it is 
clearly necessary to deliberately desynchronize the TCLs 
without interrupting their operation. However, a central- 
ized algorithm for doing so is undesirable; it is considered 
infeasible to keep close track of the internal states (tem- 
peratures, set points etc.) of all the units in a centralized 
manner, since doing so would require regular measurement 
feedback from all the devices, which would give rise to a 
very heavy communication and computational load on the 
system. 



In the following, we present a completely decentralized 
algorithm that is able to reduce the power oscillations 
discussed above without violating any of the requirements 
R1-R3. The only significant assumption we require to be 
satisfied is that the individual TCLs have access to time 
stamps of changes in the total power consumption. While 
this may be considered a form of global information, its 
usage is limited to identifying the times at which the im- 
mediate predecessor and the immediate successor of a local 
TCL changes state. It is not used to actually identify said 
units; i.e., we only require local temporal information. The 
information about changes in global power consumption 
may either be measured locally or broadcast from the 
utility, thus avoiding violating requirements Rl and R2. 

The algorithm comprises two main components. Firstly, 
the algorithm is reset whenever each TCL in question 
receives a setpoint change Osp.i 0sp,i + S. At this point, 
each TCL immediately narrows its operation interval by a 
random value a G [0, A/2], causing the dynamics of unit i 
momentarily to be governed by the dynamics 

0^{t) = -^(0oo,» - 0^{t) - S^{t)R^P^), 

{0 if s,{t-) = 1 A e,{t) < +5 + ^ -a, 

1 if s,(<-) = OA^,(t) >0,p,, + + 

Si{t~) otherwise 

This operation rapidly induces a large degree of desynchro- 
nization among the population of TCLs without violating 
requirement R3, as illustrated in Figure 4. However, ran- 
domly choosing the temperature bounds may yield very 
tight bounds, which in turn may give rise to constant 
rapid on/off-switching. This is generally not desirable, so 
we drive the bounds back to their original settings by 
introducing the dynamics 

a{t) = -aa[t) (4) 

for some appropriately chosen a > 0, as also illustrated in 
the figure. 

Now, even though most of the synchronization among 
the population of TCLs is likely to have disappeared 
with the random shrinking of the operating interval, it 
may still happen that some TCLs remain synchronized. 
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Fig. 4. Illustration of TCL desynchronization algorithm 
before and after a step occurring at i — tg. The 
temperature profiles 9i of three TCLs indexed by 
i = 1,2,3 are shown with full lines. The dashed lines 
indicate the operating intervals [^^sp,j + 6 + /S.i/2 — 
ai{t),9s-p.i + 5^ ^il'2- + oii{t)]. Due to different random 
values of a^, the TCLs become desynchronized after 
the step even though they were completely synchro- 
nized before and immediately after ts- The operation 
intervals converge exponentially back to their original 
values. 

or close to synchronized. The second component of the 
proposed algorithm is designed to deal with any 'left- 
over' synchronization among TCLs that arc close to being 
identical. Here, we make use of the fact that the behavior 
of the TCLs is periodic in steady state operation. Let 
the period be denoted T. Essentially, this part of the 
desynchronization algorithm repeatedly forces the devices 
to switch status at some point within the interval [0,T], 
and adaptively adjusts this enforced switch timing to 
achieve maximal spread across the period; see Figure 5. 

In short, each TCL records the switch timings of the 
immediately preceding and the immediately succeeding 
TCLs; then it sets the new enforced switch time to the 
mean of the switch times of the succeeding and preceding 
TCLs, i.e., 



tk+l 



iprev ~t~ ^prev 



(5) 



where f^+i e [0, T] denotes the time of the enforced switch 
within the following period, modulo T. 

The enforced switch timings are initially distributed ran- 
domly; the TCL which randomly obtains the smallest 
value will then not be able detect a switch prior to its 
own. When it recognizes this fact, it sets its own switch 
timing to and maintains it at that value (modulo T); it 
is thus not necessary to assign a global label to the first 
and/or last TCL a priori. 

Let x^. denote the time of the j'th forced transition, 
modulo T, at iteration k; that is, 



< 



< 



< T. We gather these switch timings in a vector 
Xfc G [0,T]^.We can then prove the following asymptotic 
result: 

Theorem 1. If all switch timings for i = 1,...,N are 
updated according to (5), the globally attracting switch 
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Fig. 5. Illustration of TCL desynchronization algorithm. 

The top figure shows the temperature profile of a 
TCL with (thick full line) and without (thick dashed 
line) enforced switching at time t = t„. The thin 
dashed lines show parts of the temperature traces 
of the preceding and succeeding TCLs. Their switch 
times are recorded by the local TCL as tprov and tncxt, 
respectively. The bottom figure shows the relevant 
part of the corresponding power traces. 

timing state x* (modulo T) is described by 

T{N-1) 



X* mod T 



T 2T 



N 



(6) 



Proof: The algorithm described above for the evolution of 
the state (time) dynamics requires that each state be the 
mean value of the prior and next state with the caveat that 
the TV'th element should update its value based on the next 
pulse i.e. the value of the first state element in the next 
time instant. Hence, we first extend the state vector by 
placing the new value of the first state into the fictitious 
A'' -|- 1 th dimension of the state vector x to generate a 
new state vector, denoted x. The update dynamics of the 
enforced switch times can be written as 



10 

2 1 2 1 
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.0 

= rxfe. 

which implies that 



01 






1. 



Xfe 



(7) 



lim x/j = lim F'^xq 

fc— >cx) k—>oo 

Using eigenvalue decomposition, we may write the afore- 
mentioned limit as 

1 fc 



lim Xfc = lim EA E Xq 



(8) 



where A G r(^+i)x(^+i) is a diagonal matrix of eigen- 
values of r and e R(^+i)x(^+i) contains the corre- 
sponding eigenvectors. F has two repeated unit eigenvalues 
with linearly independent eigenvectors 7 and 1 — 7, where 
1 G R^+^ is a vector of one-elements and 

1 T 



1 



1 1 







(9) 



1 2 

TV ~ TV 

The remaining eigenvalues are of absolute magnitude less 
than 1, implying that all except two of the diagonal 
elements in the eigenvalue matrix A*"' tend to zero as 
fc — > 00. Hence, in order to obtain the limiting value of the 
state dynamics, it remains only to utilize the eigenvectors 
corresponding to the unit eigenvalues to obtain the limit 
in (8). 

Partition the eigenvector matrices as follows 

1 T 

E =[ei 62 • • • CAT+i] , =[ei 62 ■■■ eN+i] 

and let ei and cn+i denote the eigenvectors corresponding 
to the unit eigenvalues, i.e., Ci = 7, Cn+i = 1 — 7. It is 
now possible to verify from the structure of T and the 
partitioning of E and E~^ that 
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1], (10) 



Finally, substituting ei — 7, e^+i ~ 1 
hm F'^' = [7 • • • 1 - 



- 7 into (8) we get 
-7]. (11) 



Thus it is seen that the asymptotic limit of the state is 
dependent only on the first and last elements of the state 
vector X. As these are fixed at and T respectively, the 
statement of the theorem follows as desired. < 
It is of interest to note that if F had been a positive (or 
irreducible) matrix, the Perron-Frobenuis theorem would 
have been an elegant approach to obtain the steady state 
matrix directly. However, in this case, owing to the zero 
entires and the lack of strong connectivity of the graph 
associated with F (a necessary and sufficient condition for 
irreducibility) , the above more involved proof is required. 

Finally, we state the full desynchronization algorithm as 
Algorithm 1, where we use the notation rand G [0,T] to 
denote the operation of picking a random value from a 
uniform distribution over the interval [0,T]. 

4. SIMULATION EXAMPLES 

Armed with Algorithm 1, we re- visit the simulation ex- 
amples in Section 2. Figure 6 shows the temperature 
curves of a subset of a population of 10,000 identical 
TCLs, simulated under the same conditions as above, i.e., 
R = 2°C/kW, C = 5 kWh/°C, P = 14 kW, 77 = 2.5, 
6»3p = 20°C, A = 1°C and Ooo = 28°C. The top subplot 
shows the behavior without desynchronization, while the 
bottom plot shows the behavior with Algorithm 1 applied. 
The temperature curves clearly look more 'jumbled' after 
the step, which indicates a greater degree of desynchro- 
nization, and the operation intervals can be seen to return 
to their original size after in the space of a few hours. 

Figure 7 shows the corresponding power curves. The 
effect of the desynchronization is very obvious almost 
immediately after the step, as the amplitude of already the 
first peak is less than the case without desynchronization, 
and most of the oscillations are suppressed after about 



Algorithm 1: Desynchronize Cooling TCL 

Data: Broadcast signal indicating a setpoint change 5] 
Initial operation setpoint 6sp and interval width 
A; initial on/off status s G {0, 1}; on/off period 
time T; constant a > 

Result: yielding maximal desynchronization for 

fc — > CXD 



begin 



i — rand G [0, T] ; 
a i — rand G [0, A/2] ; 
k< — 0; 

while fc < 00 do 
for < i < T do 

Measure 0{t) ; 
ii t < tk then 

|_ tpi-cv < — time of last transition before i„ ; 

if t — tk then 
if s = then 

L 1 ; 
else 

L s ^ ; 

ii t > tk then 

|_ ^prcv < — time of first transition after t„ ; 
if 9{t) <0sp - ^ + a then 



^sp 2 

L .s«(i) ^ ; 
else if 9{t) > O^p 

L Mt) ^ 1 ; 

if s = 1 then 

|_ Turn cooling power on 



A — a then 



a 
t i 



- ae " 
t + e 



tk+i ^ (t pre 
k i — fc + 1 ; 
t< — ; 



+ tnoxt)/2 ; 





Fig. 6. Simulation with 10,000 identical TCLs subjected 
to a step change at time t = 10 h. Top: no desynchro- 
nization; bottom: with synchronization 

one period. We also notice that there are some oscillations 
remaining, which will gradually die out due to the long- 
term behavior of the adaptive algorithm. 




Fig. 7. Simulation with 10,000 identical TCLs subjected 
to a step change at time t = 10 h. Full line: with 
desynchronization; dashed: without synchronization 




Fig. 8. Simulation with 10,000 inhomogeneous TCLs sub- 
jected to a step change at time t = lOh. Full line: with 
desynchronization; dashed: without synchronization 

Figure 8 shows a simulation with a heterogenous popula- 
tion of TCLs. Here, as opposed to Figure 7, the parasitic 
oscillations die out in the case without desynchroniza- 
tion as well. However, Algorithm 1 clearly speeds up the 
process, however. Note that the oscillations that can be 
seen before the setpoint change is due to the initialization 
of the TCL on/off states, which does not match exactly 
with the steady-state distribution (the desynchronization 
is not active before t = 10 h). Note also that, by changing 
the setpoint of all the TCLs in the simulation by half a 
degree, it was possible to reduce the mean total power 
consumption by approximately 2 MW after the step. 

5. DISCUSSION 

This paper presented a novel algorithm for counter-acting 
unwanted oscillations caused by synchronization of pop- 
ulations of temperature controlled loads, which requires 
neither central management of the individual units nor 
communication between units. The algorithm comprises 
two main part, a 'fast' randomization of the tempera- 
ture bands within which the TCLs operate, and a 'slow' 
adaptive adjustment of enforced switch timings, which 
maximizes desynchronization over time. We presented a 
formal proof of convergence of homogeneous populations 
to the desynchronized status, as well as simulations that 
indicate that the algorithm is able to effectively dampen 



power consumption oscillations for both homogeneous and 
heterogeneous populations. 

However, the simulations also indicate that even with ac- 
tive desynchronization, it is hard to avoid large peaks right 
after a common broadcast setpoint step. Thus, high-level 
model-based control along the lines of the work presented 
in Kundu et al. [2011], will probably be preferable to 
simple steps. 
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